This application claims the priority of German Application No. 198 03 158.0, filed Jan. 28, 1998, the disclosure of which is expressly incorporated by reference herein.
The invention relates to an apparatus for determining the alertness of a person, especially a person operating a motor vehicle or machine, having an imaging system for imaging the area of at least one eye of the person, and an image evaluating system which contains eye closing detection means and serves to evaluate the images detected by the imaging system. Such an apparatus serves, for example, to continually monitor the alertness of a driver and to warn him while he is driving and before he falls asleep. For this purpose, pictures of the area of at least one eye are taken by the image pickup system and analyzed by an image evaluating system.
German patent applications DE 196 21 435 A1 and DE 197 15 519 A1, as well as in U.S. Pat. Nos. 4,359,724, 5,402,109 and 5,566,067 disclose an apparatus of this kind. The image evaluating system contains means for recognizing the closing of the eye, whereby any increase in the length of the closing of the eyes is interpreted as an increase of fatigue.
In the system disclosed in DE 196 21 435 A1, the ability to measure pupil movement can also be provided (especially the speed of the pupil). This is used as an additional parameter for the decision as to whether the observed person is awake or sleepy. In the system disclosed in DE 197 15 519 A1, by which the sleepiness of a driver is to be estimated, in an initial period of time after the vehicle is started, a frequency distribution of the blink time of the eye of the driver is established. From this frequency distribution, a threshold value is derived, which is then used in an adjoining, second period of time for judging the degree of sleepiness of the driver. From this, a threshold value is derived which is then employed in a directly following second period of time for judging the degree of sleepiness. This procedure is intended to compensate for differences in the blink period and blink frequency in different people. Here, after starting the vehicle, it is assumed that the driver has a low level of drowsiness.
Picture taking and evaluating systems having means for determining pupil diameter and/or eye movement (i.e., pupil movement) are known from the science of pupillography. They are used in eye research for detecting a pupil's diameter and movements of the eyes. Systems of this kind are disclosed in German patent applications DE 35 41 726 A1, DE 44 19 489 A1 and DE 41 40 160 A1 U.S. Pat. Nos. 3,966,310, 4,755,043, 4,850,691, 5,187,506 and 5,204,703.
Furthermore, it is known that fatigue can be detected from the variation of the pupil's diameter. Studies show especially that, with increasing fatigue the pupils at first narrow, even if the ambient brightness is not especially great, and then fluctuations of the diameter of the pupils occur. This indicates the feasibility of using pupillography to identify persons who are in danger of falling asleep, for example, during road, sea and air travel; see B. Wilhelm, H. Wilhelm, "Die Pupille als Schlaf-Wach-Indikator" in Z. prakt. Augenheilkd. 15, 1994, pages 185, and I. E. Loewenfeld, "The Pupil," vol. I, Wayne State University Press, Detroit, 1993.
The object of the present invention is to provide an alertness detecting apparatus, whereby the wakefulness or alertness of a person (especially the operator of a motor vehicle or of machinery) can be especially reliably determined in order to produce therefrom appropriate responses, e.g., issuance of a timely warning or placing the system being steered or operated by the person into a harmless state.
This and other objects and advantages are achieved by an alertness detecting apparatus having an imaging system for imaging the area of at least one eye of the person, and an image evaluating system which contains eye closing detection means and serves to evaluate the images detected by the imaging system. In this apparatus, the image evaluating system has, in addition to eyelid analyzing means, means for determining pupil diameter as well as an evaluation system which determines alertness in accordance with the closed-eye information obtained by the closed-eye detection means. Once the pupil diameter information is obtained by the pupil diameter determining means, the detected alertness of the driver is classified accordingly into one of at least three stages. Due to the presence of both the closed eye detection means and the pupil diameter determining means, a certain redundancy is achieved, as well as a high reliability in the determination of the state of alertness. Classifying the driver's state of alertness in three or more levels creates the possibility of reacting in stages to incipient fatigue of the person being observed.
In an embodiment of the apparatus according to the invention, the image evaluating system contains means for determining the ambient brightness, and the evaluating system has a correlation unit by which it is possible to determine the degree of correlation of the size of the pupil with the ambient brightness. In this manner, changes in pupil diameter due to fluctuations of the ambient brightness can be accurately distinguished from pupil diameter variations due to the level of alertness.
An embodiment of the apparatus is designed such that, with the multi-stage determination of alertness, it will regard the state of alertness to be highest (i.e., the wide-awake state) if that state is indicated both by a brief closing of the eye, a comparatively high closed-eye frequency and the absence of pupil size variations which do not correlate with the ambient light. On the other hand, the presence of the lowest state of alertness (i.e., the highest degree of fatigue) is assumed whenever the observed duration of the closed-eye times are excessively long.
An embodiment of the apparatus permits an at least four-step determination of alertness. Here, in addition to the stages of highest and lowest alertness, a first intermediate stage of greater alertness and a second intermediate state of lower alertness are defined. These are defined according to whether the measured pupil size fluctuations are below (in one case) and above (in the other case) an appropriate threshold value, and furthermore at least one of the conditions is fulfilled. These conditions are that the pupil size fluctuations do not correlate well with ambient light fluctuations, or the closed-eye duration is in a middle range, or the blinking frequency is below an appropriate threshold value. This achieves a very sensitively graded recognition of fatigue.
An embodiment of the apparatus according to the invention contains a warning unit, actuated by the image evaluation system, which emits warning signals which correspond to the alertness level (locally), especially to the person being observed, or also transmits a signal to a point remote from the person being observed. The transmission can occur from a vehicle to a central station or to other vehicles, for example.
In an embodiment of the apparatus according to the invention, a warning unit is provided which operates a vehicle or machine control system according to the degree of alertness detected. This occurs in such a way that, when a sufficiently low alertness of the person being observed is detected, the vehicle being driven or the machine being operated by the driver will be brought by the control system to a safe operating state. This actively forestalls accidents in the case of vehicles that might be due to driver fatigue, for example, by safely slowing the vehicle or bringing it to a stop through an automatic vehicle driving system activated by the warning unit.
In an embodiment of the apparatus according to the invention, upon recognition of a sufficiently low state of alertness, a warning unit causes an operating unit to give the person under observation an alertness test. Other system reactions can then be made to depend on the proper execution of the alertness test.
Other objects, advantages and novel features of the present invention will become apparent from the following detailed description of the invention when considered in conjunction with the accompanying drawings.